SHAP and
LIME to asses the influence of demographic variables on
model performance.Consider a binary variable for gender, \(z\) then:
\[ f(\vec{x}) = f(\vec{x}, z) \]
This means: \(P(\hat{y} = 1 | z = 0, y = 1) = P(\hat{y} = 1 | z = 1, y = 1)\)
“Intersectionality means that people can be subject to multiple, overlapping forms of oppression, which interact and intersect with each other.” - Kimberly Crenshaw
Model explainability is a concept which looks into the ability to understand the results of a machine learning model.
In 2007 a teacher was fired from a Washington DC school due to an algorithm: Despite having highly favourable reviews from students and parents, an opaque algorithm was used to determine her performance as being in the bottom 2% of all teachers.
Woohoo! We’ve trained up a model, evaluated that it is working effectively, and completed the R&D process, our models are then deployed into production.